An Augmented Reality Based User Interface Model for Full Body Interaction using Monocular 3D Pose Estimation

In this paper, we present a conceptual design of augmented reality full body interaction based on monocular 3D pose estimation. The proposed design is based on 3D pose estimation from the image of user’s motions captured by a monocular camera and the processing of 3D human poses for augmented reality applications. Based on the method, a 3D human full body model is constructed. The silhouettes extraction from the image captured is being matched with the projection on a virtual image plane. Multipart alignment is used to adjust 3D poses of the graphical 3D human full body based on silhouettes extraction. Each of body part alignment will be used to define a set of vector of body parts represented as a skeleton model. The obtained model is used to identify the human pose and the associated 3D motion parameters. The vector for each body part is used as 3D pose information in the proposed augmented reality system to achieve the marker-free interaction in the augmented environment. This method will be applied for interaction in augmented reality (AR) environment.